[FEATURE] Command to provide information for OCaml syntax #1706

PizieDust · 2023-11-15T22:46:02Z

FEATURE: New Merlin Command - Syntax Document

Introduction

The Document command of Merlin returns comments written in code. This PR introduces a new command, the Syntax Document command which takes the syntax under the cursor, analyses it's TypedTree nodes and returns specific documentation for the Syntax.

Code Example

Code

type point = { x:int; y:int }

Output

{
   "name": "Record types",
   "description": "Allows you to define variants with a fixed set of fields, and all of the constructors for a record variant type must have the same fields",
   "url": "https://v2.ocaml.org/manual/extensiblevariants.html"
}

This command can also identify specific

Code

type point = private { x:int; y:int }

Output

{
   "name": "Private Record Types",
   "description": "Values of a record type declared private can be de-structured via the expr . field notation. However, values of these types cannot be constructed directly by record construction.",
   "url": "https://v2.ocaml.org/releases/5.1/htmlman/privatetypes.html#ss:private-types-variant"
}

Scope

This command for a start covers Language Extensions of OCaml. The covered sections include:

More sections will be covered and updated to the command along with other parts of the OCaml programming language.

cc @voodoos @pitag-ha

src/analysis/syntax_doc.ml

src/frontend/ocamlmerlin/query_json.ml

src/frontend/query_commands.ml

voodoos

That's much better !

Now is probably a good time to decide which features we actually want to document.

There are two possibilities here:

If the feature is aimed at complete beginner, it might make sense to document as much of the syntax as we can.
Or we just want to shed a light on weird / rarely use OCaml features that might puzzle even seasoned developers.

@trefis, @pitag-ha, what's your feeling about this ?

It might also depend on how the feature is used: is it called on every on-hover in LSP ? Or only on specific code actions ?

src/analysis/syntax_doc.ml

voodoos · 2023-11-23T12:38:47Z

@PizieDust we discussed with trefis and Sonja and for the scope of the feature we think you can first focus on "language extensions". These are advanced features that were added to OCaml other the time and are often not well known by beginners.

You can find the list of these features here:
https://v2.ocaml.org/releases/5.1/htmlman/extn.html

About the UI, we can start (after the backend part in Merlin is done) by extending hover with the new information and then make it configurable. (And in fact, rework the hover command in general, it could be more useful than it is now in some situations).

PizieDust · 2023-11-23T13:41:10Z

Amazing. I should be moving this PR from draft soon

voodoos

I made a (very incomplete) round of review: could you reformat the file (I provided the result of ocamlformating it in a comment), and refactor / promote the result of the tests so that they can be used to understand the expected behavior ?

tests/test-dirs/syntax-document/language-extensions.t/run.t

src/frontend/query_protocol.ml

src/analysis/syntax_doc.ml

pitag-ha

This is super cool, @PizieDust , thanks a lot!

Are you planning to add an mli-file for syntax_doc.ml once you won't change that file that much anymore?

src/analysis/syntax_doc.ml

voodoos

Nice, you're making good progress on this :-)

I made a few comments, and you should also try to sort out the rebase mess :-)

src/analysis/syntax_doc.ml

src/frontend/query_protocol.ml

tests/test-dirs/syntax-document/language-extensions.t/first-class-modules.ml

src/analysis/syntax_doc.ml

voodoos

Right, I think we just need to improve the locally abstract data types case before merging a first version of this PR.

src/analysis/syntax_doc.ml

tests/test-dirs/syntax-document/language-extensions.t/run.t

voodoos

I think the PR is almost ready.
I just made a few formatting remarks.

Do you want to rebase and squash your changes into meaningful commits ?
If not I can simply squash everything before merging.

src/frontend/ocamlmerlin/query_json.ml

src/frontend/query_commands.ml

CHANGES.md

* add new command boilerplate * add identifier to syntax_doc definition * update query json to use new identifier * include new return types for query protocol * update test to check type variant declarations * syntax documentation boiler plate * poc implementation for syntax_doc command * update the test description * remove optional identifier and obsolete code * remove optional identifier * add new variant examples * handle type declarations * remove redundant parent_node matching * remove comments * Bump version for release 4.13 * delete intial testing file * add custom documentation * add language extension tests * refined node matching for better docs * add more tests * remove invalid_identifier ouput * ocamlformat, limit lines to 80 chars * add mli for syntax_doc * proper naming * update docs, move type to query_protocol, leave as record * add info type for syntax_docment command json * update to record for json output * change from string output to json * delete redundant test files * delete tests files * move tests code here and promote * remove typedtree nodes * change variable name to a more informative name * Make syntax_doc_result optional * remove redundant cases * use singular and more shorter names * dune promote name changes * lint * url builder function for syntax documentation url * lint * dune promote correct urls * concat urls before returning to query_json * make command return record option * delete test file * correct formatting to original * refactor to be more meaningful * use versbose names * test: start making more precise tests * Apply suggestions from code review Co-authored-by: Ulysse <[email protected]> * Edit descriptions to be less verbose * dune promote description changes * use syn_doc alias * merge duplicate case results * lint * reduce verboseness in test * add eof * Update src/frontend/ocamlmerlin/new/new_commands.ml Co-authored-by: Ulysse <[email protected]> * cover more test * update some match cases * remove trailing whitespaces * use plural form * refactor private and public types for same nodes into one match case * lint and seperate abstract types for public and private * better targeting first class modules * more test for first class modules and capitalization corrections * pass location position to syntaxdoc logic * use cursor position to better target locally abstract datatypes * test case where locally abstract dt shouldnt be triggered * test for first class module where it shouldnt be triggered * use 3rd person singular * add changelog * fix indentation * remove excess whitespace * proper heading --------- Co-authored-by: Ulysse Gérard <[email protected]> Co-authored-by: Ulysse <[email protected]>

CHANGES: Thu Feb 22 14:00:42 CET 2024 + merlin binary - Add a "heap_mbytes" field to Merlin server responses to report heap usage (ocaml/merlin#1717) - Add cache stats to telemetry (ocaml/merlin#1711) - Add new SyntaxDocument command to find information about the node under the cursor (ocaml/merlin#1706) - Fix `FLG -pp ppx.exe -as-pp/-dump-ast` use of invalid shell redirection when direct process launch on Windows. (ocaml/merlin#1723, fixes ocaml/merlin#1722) - Add a query_num field to the `ocamlmerlin` responses to detect server crashes (ocaml/merlin#1716) - Jump to cases within a match statement (ocaml/merlin#1726) - Jump to `module-type` (ocaml/merlin#1728, partially fixes ocaml/merlin#1656) - Exposes stable functions for configuration handling and pattern variable destruction. (ocaml/merlin#1730) + editor modes - vim: load merlin under the ocamlinterface and ocamllex filetypes (ocaml/merlin#1340) - Fix merlinpp not using binary file open (ocaml/merlin#1725, fixes ocaml/merlin#1724)

voodoos reviewed Nov 22, 2023

View reviewed changes

PizieDust changed the title ~~add new command boilerplate~~ [NEW FEATURE] Add documentation for OCaml syntax Nov 22, 2023

PizieDust changed the title ~~[NEW FEATURE] Add documentation for OCaml syntax~~ [NEW FEATURE] Add command to provide information for OCaml syntax Nov 22, 2023

PizieDust requested a review from voodoos November 22, 2023 14:30

voodoos reviewed Nov 22, 2023

View reviewed changes

src/analysis/syntax_doc.ml Outdated Show resolved Hide resolved

PizieDust mentioned this pull request Dec 11, 2023

Demystifying OCaml Syntax with Merlin and OCaml-LSP ocaml/ocaml-lsp#1218

Merged

PizieDust marked this pull request as ready for review December 11, 2023 11:44

PizieDust requested a review from voodoos December 11, 2023 12:14

voodoos reviewed Dec 11, 2023

View reviewed changes

tests/test-dirs/syntax-document/language-extensions.t/run.t Outdated Show resolved Hide resolved

src/frontend/query_protocol.ml Outdated Show resolved Hide resolved

src/analysis/syntax_doc.ml Outdated Show resolved Hide resolved

pitag-ha reviewed Dec 11, 2023

View reviewed changes

src/analysis/syntax_doc.ml Outdated Show resolved Hide resolved

voodoos reviewed Dec 12, 2023

View reviewed changes

src/analysis/syntax_doc.ml Outdated Show resolved Hide resolved

PizieDust changed the title ~~[NEW FEATURE] Add command to provide information for OCaml syntax~~ [FEATURE] Command to provide information for OCaml syntax Dec 15, 2023

PizieDust requested review from pitag-ha and voodoos December 18, 2023 05:41

voodoos reviewed Dec 18, 2023

View reviewed changes

PizieDust added 13 commits December 18, 2023 15:43

add new command boilerplate

d94449b

add identifier to syntax_doc definition

bd358ad

update query json to use new identifier

064be5d

include new return types for query protocol

e839a74

update test to check type variant declarations

fffcb72

syntax documentation boiler plate

854dbc7

poc implementation for syntax_doc command

395908a

update the test description

7d260d7

remove optional identifier and obsolete code

b7a58c3

remove optional identifier

ee2e77a

add new variant examples

a1b83cf

handle type declarations

7637d07

remove redundant parent_node matching

4465115

PizieDust added 4 commits January 30, 2024 04:52

refactor private and public types for same nodes into one match case

5872565

lint and seperate abstract types for public and private

5221a23

better targeting first class modules

9f104b6

more test for first class modules and capitalization corrections

fb2f278

PizieDust requested a review from voodoos January 31, 2024 08:54

voodoos reviewed Jan 31, 2024

View reviewed changes

src/analysis/syntax_doc.ml Outdated Show resolved Hide resolved

voodoos reviewed Jan 31, 2024

View reviewed changes

src/analysis/syntax_doc.ml Outdated Show resolved Hide resolved

tests/test-dirs/syntax-document/language-extensions.t/run.t Outdated Show resolved Hide resolved

PizieDust added 5 commits January 31, 2024 21:05

pass location position to syntaxdoc logic

adf294d

use cursor position to better target locally abstract datatypes

d15324c

test case where locally abstract dt shouldnt be triggered

82997fd

test for first class module where it shouldnt be triggered

29d1c26

use 3rd person singular

c726db5

PizieDust requested a review from voodoos January 31, 2024 20:17

PizieDust and others added 2 commits February 1, 2024 06:26

add changelog

b92e9bb

Merge branch 'master' into syntax_documentation

964b183

voodoos approved these changes Feb 2, 2024

View reviewed changes

src/frontend/ocamlmerlin/query_json.ml Outdated Show resolved Hide resolved

src/frontend/query_commands.ml Show resolved Hide resolved

CHANGES.md Outdated Show resolved Hide resolved

PizieDust added 3 commits February 2, 2024 13:37

fix indentation

f3cd471

remove excess whitespace

b5e9189

proper heading

cece67d

voodoos merged commit 0f64255 into ocaml:master Feb 8, 2024
1 check passed

voodoos mentioned this pull request Feb 22, 2024

[new release] merlin-lib, merlin (4.14-501/500/414) ocaml/opam-repository#25315

Merged

liam923 mentioned this pull request May 23, 2024

Backport 1706 janestreet/merlin-jst#66

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] Command to provide information for OCaml syntax #1706

[FEATURE] Command to provide information for OCaml syntax #1706

PizieDust commented Nov 15, 2023 •

edited

Loading

voodoos left a comment

voodoos commented Nov 23, 2023 •

edited

Loading

PizieDust commented Nov 23, 2023

voodoos left a comment •

edited

Loading

pitag-ha left a comment

voodoos left a comment

voodoos left a comment

voodoos left a comment

[FEATURE] Command to provide information for OCaml syntax #1706

[FEATURE] Command to provide information for OCaml syntax #1706

Conversation

PizieDust commented Nov 15, 2023 • edited Loading

FEATURE: New Merlin Command - Syntax Document

Introduction

Code Example

Code

Output

Code

Output

Scope

voodoos left a comment

Choose a reason for hiding this comment

voodoos commented Nov 23, 2023 • edited Loading

PizieDust commented Nov 23, 2023

voodoos left a comment • edited Loading

Choose a reason for hiding this comment

pitag-ha left a comment

Choose a reason for hiding this comment

voodoos left a comment

Choose a reason for hiding this comment

voodoos left a comment

Choose a reason for hiding this comment

voodoos left a comment

Choose a reason for hiding this comment

PizieDust commented Nov 15, 2023 •

edited

Loading

voodoos commented Nov 23, 2023 •

edited

Loading

voodoos left a comment •

edited

Loading